Scheduling of different work units (for example, batch jobs) is a commonplace activity in data processing systems. For this purpose, workload schedulers have been proposed in the last years to automate the submission of large quantities of jobs from a central point of control; an example of commercial scheduler is the “IBM Tivoli Workload Scheduler (TWS)” by IBM Corporation.
Typically, the scheduler controls the submission of the jobs for their execution according to a specific plan. The plan defines the flow of execution of the jobs to be submitted in a production period (such as one day); particularly, the plan orders the execution of the jobs according to their dependencies (for example, based on constraints defined by the completion of other jobs or the availability of required resources of the system).
The generation of the plan is a very complex and time-consuming activity (which requires the addition and the removal of thousands of jobs). Therefore, this operation may have a detrimental impact on the workload of the system (taking into account that it must be repeated every day for lie next production period).
A solution known in the art for mitigating the above-mentioned problem is of creating a preliminary long-term plan in advance. For example, the long-term plan may be expressed in terms of streams of jobs (each one including multiple jobs treated as a single work unit); in this case, the long-term plan simply orders the execution of the streams according to their sequence dependencies (disregarding any other constraints). As a result, the complexity of the task is strongly reduced, so that the long-term plan can span over a very long period (for example, some months). Moreover, the operation is typically executed with a low frequency (for example, only when a few days remain in the future), so that it does not significantly impair the performance of the system.
On the other hand, the actual (production) plan is generated by extracting the portion of the long-term plan corresponding to the relevant production period; the streams are then expanded into the corresponding jobs, and the remaining dependencies are resolved. In this way, the time required to generate the production plan is strongly reduced.
However, the above-described solution is not completely satisfactory. Particularly, this technique requires the maintenance of two plans (i.e., the long-term plan and the production plan); this adversely affects the management of the scheduling environment.
Moreover, whenever the dependencies among the streams are changed, the corresponding execution flow in the long-term plan may be invalidated. Therefore, it is necessary to identify the streams affected by the changes in the long-term plan, and then redefine their order of execution. However, this further increases the complexity of the task of managing the long-term plan.
All of the above has a detrimental impact on the activity of planning the execution of the jobs. Particularly, the above-mentioned side effects may defeat the advantages provided by the availability of the long-term plan (for the generation of the production plan).